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Executive Summary 

National Board Certification is a voluntary process established by the National Board 
for Professional Teaching Standards (NBPTS) to measure what accomplished teachers should 
know and be able to do. Certification is achieved through a rigorous performance-based 
assessment that takes between one and three years to complete. As of November 2004, 
approximately 40,200 teachers had earned National Board Certification. 

This study was undertaken as part of the National Board’s continuing effort to 
measure the impact of National Board Certification and the effects of National Board 
Certified Teachers (NBCTs) on the quality of teaching and student achievement in America’s 
schools. End-of-grade mathematics and reading test scores from two large North Carolina 
school districts (Charlotte-Mecklenberg and Wake County) from the 1999-2000 through 
2002-2003 school years, grades 4 through 8 were analyzed to compare NBCTs with other 
teachers. Over 260,000 student records (about half in mathematics and half in reading), 
representing over 4600 teacher-subject-grade-year combinations, were included in the 
analyses. Of that 4600+, 281 represented National Board Certified mathematics teacher-years, 
306 represented National Board Certified reading teacher-years. 



' Statisticians within SAS EVAAS group, SAS Institute, Inc. 



1 




Models were fitted to eaeh of the ten subject-grade combinations (1) using end-of- 
grade scores as the response variable with end-of-grade scores from the previous year as 
covariates, and (2) using gain scores (end-of-grade score minus previous year end-of-grade 
score) as the response variable. Additional explanatory variables included: teacher 
certification status (the factor of interest), teacher years-of-experience, and the gender and 
race of the student. A hierarchical model was used to account for the fact that students were 
nested within teachers. For comparison with other recent studies, non-hierarchical models 
were fitted as well. Three planned comparisons assessed the differences between NBCTs and 
other teachers: (1) NBCTs versus teachers who have never been involved in the certification 
process, (2) NBCTs versus teachers who planned to attain certification in the future, (3) 
NBCTs versus teachers who failed in their attempt at certification. 

Findings. Overall, based on the hierarchical models, students of NBCTs did not have 
significantly better rates of academic progress than students of other teachers and estimated 
effect sizes were relatively small. The more relevant and important finding was that the 
variation among teachers within the same certification status was sufficiently large that 
whatever small average differences there were between teachers in different certification 
status categories were rather meaningless in comparison. As a result, a student randomly 
assigned to a NBCT is no more likely to get an “effective” (or an “ineffective”) teacher than a 
student assigned to a non-NBCT. 

Implications. The findings of this study do not support the conclusion that, in general, 
students of NBCTs receive better quality teaching than students of other teachers. This is in 
contrast to the findings of several other recent studies, none of which used hierarchical models 
to properly account for the nested structure of the data. Since failure to properly model 
hierarchically structured data is well-know to produce overly optimistic results, the 
conclusions from those earlier studies need to be reassessed. If the findings of this North 
Carolina study are representative of other states, the NBPTS may wish to consider what steps 
should be taken to strengthen the certification process to better assess teacher quality. 

1. Background 

National Board Certification is a voluntary process established by the National Board 
for Professional Teaching Standards (NBPTS), an independent, nonprofit, nonpartisan and 
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nongovernmental organization governed by a board of direetors, the majority of whom are 
elassroom teaehers. Its mission is to establish high and rigorous standards for what 
aoeomplished teaehers should know and be able to do. Certifieation is aehieved through a 
rigorous performanee-based assessment that takes between one and three years to eomplete. 
As of November 2004, approximately 40,200 teaehers had earned National Board 
Certifieation. 

In its eontinuing effort to measure the impaet of National Board Certifieation and the 
effeets of National Board Certified Teaehers (NBCTs) on the quality of teaehing and student 
aehievement in Ameriea’s sehools, NBPTS has engaged in an extensive, independent, and 
rigorous researeh agenda. As part of these eontinuing researeh efforts, NBPTS eontraeted the 
EVAAS® group at the SAS Institute Ine. to study the impaet of NBCTs on the rate of 
aeademie progress relative to other teaehers who were not eertified by the National Board. 

2. Data and Models 

This study addressed three primary research questions: 

1 . Do students of NBCTs make greater academic progress than students of teachers who 
have never attempted to attain National Board Certification? 

2. Do students of NBCTs make greater academic progress than students of non-NBCTs who 
plan to attempt to attain National Board Certification at some point in the future? 

3. Do students of NBCTs make greater academic progress than teachers who attempted to 
attain National Board Certification, but who failed in their first attempt? 

Data 

To test the three major researeh questions, end-of-grade test data from two large North 
Carolina sehool distriets - Charlotte -Meeklenberg and Wake County - from sehool years 
1999-2000 to 2002-2003 were analyzed. North Carolina has eonsistently ranked first 
nationwide in terms of numbers of NBCTs, and both distriets have substantial numbers of 
sueh teaehers. 
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Multiple years of data for students in grades four through eight in reading and 
mathematies were analyzed. Multiple eohorts of data for eaeh grade and subjeet were used 
in the analyses. Sample sizes of teaehers and students varied by grade and by subjeet, with the 
numbers of eertified teaeher-years ranging from 13 to 122, and the number of students 
ranging from about 16,000 to 37,000, aeross grades and subjeets (see Table 1). A student’s 
reeord was exeluded from the study when one or more of the following eonditions were 
present: 

• A linkage eould not be established from a teaeher and a student reeord; 

• the student reeord laeked two prior year seores; 

• the student was taught in a elassroom having 10 or fewer students; 

• the raee and gender of the student were not known; 

• the teaeher’s experienee eould not be established; 

• and the teaeher withdrew from the NBPTS program. 

More than 200,000 student reeords were examined in reading and mathematics across the five 
grade levels. Following exclusions for reasons cited above, more than 130,000 records in 
each subject were used in this study. 

Models 

The study addressed each of the three primary research questions through four 
different models. These models form a two-by-two array based on (a) which response variable 
was used and (b) whether or not a random effect for teacher-within-certification-status was 
included. 



Model 


ResponseVariable 


Fixed Effects 


Random Effects 


1 


Current Student Test Score 


Year, Prev scores(year), Race, Sex, Teacher Years 
Experience, NBPTS Certification Status 




2 


Current Student Test Score 


Year, Prev scores(year). Race, Sex, Teacher Years 
Experience, NBPTS Certification Status 


Teacher(NBPTS 
Certification Status) 



^ In 2003 the North Carolina Department of Public Instruction changed its scaling procedures for the state’s end- 
of-grade tests. For the purposes of these analyses, all pre-2003 test data were mapped using translation tables 
supplied by the Department of Public Instruction. 
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3 


Simple Gain 


Year, , Race, Sex, Teacher Years 

Experience, NBPTS Certification Status 




4 


Simple Gain 


Year, , Race, Sex, Teacher Years 

Experience, NBPTS Certification Status 


Teacher(NBPTS 
Certification Status) 



Models 1 and 2 used the current student test score as the response variable with 
previous year math and reading scores included as fixed effect covariates. Models 3 and 4 
used simple gain as the response variable and did not include any previous test scores as 
covariates. Models 1 and 3 did not include a teacher random effect while models 2 and 4 did 
include this effect. Additional fixed effects were included in all models as shown in the table 
above. 

In all four models, the effect of interest was the teacher’s NBPTS Certification Status, 
with four categories: (1) teachers who had already earned National Board Certification, (2) 
those who would attempt to obtain National Board Certification in the future, (3) those who 
attempted and failed to obtain National Board Certification, and (4) those who had never 
participated in the National Board Certification process. 

Models 1 and 3 were included primarily for comparison to earlier studies. Model 1 is 
similar to that of Cavalluzzo (2004) in the use of the current test score as the response with a 
previous test score as a covariate. Model 3 is similar to the model used by Goldhaber and 
Anthony (2004). See Section 4 for more about the findings in these studies. 

Models 2 and 4 are analogous to Models 1 and 3, respectively, but also include a 
random teacher effect with a separate variance component for each certification status. These 
random effects are used recognizing the hierarchical structure of the data, capturing any 
systematic difference in performance which is shared by students linked to a particular 
teacher, but which is unrelated to any measured teacher or student characteristics already in 
the models. Failure to properly account for the hierarchical structure of the data typically 
produces inferences that are overly optimistic, and it will be seen below that 
inclusion/exclusion of the teacher random effect has a dramatic impact on the inferential 
conclusions that can be drawn from this study. These differences will be explored in some 
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detail in the Diseussion section after the Findings are presented. Note that neither the 
Cavalluzzo (2004) study nor the Goldhaber and Anthony (2004) study included a random 
effect for teacher. 

Each of the four models was run separately for each subject (Math and Reading) and 
for each grade (4, 5, 6, 7, 8) for a total of ten runs of each model. In each model three planned 
comparisons were made, comparing Certified teachers to each of the other three categories. 
Thus, in each model a total of 30 comparisons was made (3 comparisons in each of the 10 
subject-grade combinations). 



3. Findings 

The primary research questions in this study involved comparison of NBPTS certified 
teachers versus other teachers (see Section 2). These questions are addressed in the following 
paragraphs with further discussion in Section 4. Results are summarized in Table 2 

In Models 1 and 3 students of NBCTs tended to have better average performance, 
either in terms of larger end-of-grade test scores conditional on prior scores (Model 1) or in 
terms of larger gain scores (Model 3), than students from other teacher categories (see Table 2 
and Tables 3A and 3C). There were more statistically significant effects in mathematics than 
reading with these models (Math: 9 of 15 with Model 1, 8 of 15 with Model 3; Reading: 4 of 
15 with Model 1 one of which was negative!, 2 of 15 with Model 3 one of which was 
negative). The sizes of the effects were generally less than one-half of a scale score unit and 
translated to standardized effect sizes that averaged 0.09 and 0.04 for math and reading, 
respectively, in Model 1, and 0.06 and 0.02 in Model 3. These effect sizes are roughly 
consistent with those reported by Cavalluzzo (2004) and by Goldhaber and Anthony (2004). 

On the other hand. Models 2 and 4, which included teacher random effects, found 
substantially fewer statistically significant effects (see Tables 2, 3B, 3D), particularly in 
mathematics (Math: 1 of 15 with Model 2, 1 of 15 with Model 4; Reading: 3 of 15 with 
Model 2, 1 of 15 with Model 4 and it was negative). Standardized effect sizes average 0.07 
and 0.04 for math and reading, respectively, in Model 2, and 0.05 and 0.01 in Model 4. 

The dramatic difference in results between Model 1 and Model 2 (or between Model 3 
and Model 4) is discussed further in the following section. It is important, however, not to let 
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that discussion distract from what is arguably the most important finding of this study: the 
amount of variability among teachers with the same NBPTS Certification Status is 
considerably larger than the differences between teachers of different Status. 

Consequently, a student who is randomly assigned to a National Board Certified teaeher is not 
mueh more likely to get an “effeetive” teaeher (or an “ineffeetive” teaeher) than a student 
assigned to a teaeher who has never been in the NBPTS proeess (or one who failed 
eertifieation, or one who may in the future beeome eertified). 

Figures 1 through 10 show the estimated teaeher effeets obtained using a model 
similar to Model 2 but with the fixed effeet for NBPTS Certifieation Status omitted from the 
model. Teaehers planning to be eertified in the future were omitted from the plots to avoid 
elutter. As an example, eonsider Figure 2 (5**' grade math). In Model 1 for 5* grade math 
(whieh exeluded teaeher effeets), NBCTs were found to be signifieantly better than other 
teaehers while in Model 2 (with teaeher effeets) the eomparisons were mostly not signifieant 
(see Table 2). What Figure 2 shows is the tremendous amount of overlap among teaehers of 
different eertifieation status. Several of the best teaehers (largest positive teaeher effeets) were 
NBCTs, but so were some of the least effeetive teaehers. Likewise, among teaehers who tried 
and failed to aehieve eertifieation, and among teaehers never involved in the eertifieation 
proeess, were some of the most and least effeetive teaehers. The ability of the eertifieation 
proeess to distinguish effeetive from ineffeetive teaehers is weak at best, with a high 
probability of miselassifieation. 



4. Discussion 

Although the findings of Models 1 and 3 versus 2 and 4 might appear eontradietory, 
there is a simple but very important explanation for the differenees. We foeus on Models 1 
and 2 in the diseussion for elarity. By ineluding teaeher random effeets. Model 2 aeeounts for 
the hierarehieal strueture of the data (students nested within teaehers), whieh is ignored in 
Model 1 . A wealth of literature exists attesting to the danger of ignoring the nested (or 
elustered or hierarehieal) strueture of data sets sueh as the one analyzed in this study. One 
useful resouree is The Centre for Multilevel Modelling (www.mlwin.eom) direeted by 
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professor Harvey Goldstein. Beeause of the importanee of this issue in the present study, it 
seems appropriate to quote at length from this website’s Introduction to Multilevel Modelling. 



Many kinds of data, including observational data collected in the human and 
biological sciences, have a hierarchical or clustered structure. . . . 

To ignore this [hierarchical structure] risks overlooking the importance of group 
effects, and may also render invalid many of the traditional statistical analysis 
techniques used for studying data relationships. 

A simple example will show its importance. A well known and influential study of 
primary (elementary) school children carried out in the 1970's (Bennett, 1976) claimed 
that children exposed to so called 'formal' styles of teaching reading exhibited more 
progress than those who were not. The data were analysed using traditional multiple 
regression techniques which recognised only the individual children as the units of 
analysis and ignored their groupings within teachers and into classes. The results were 
statistically significant. Subsequently, Aitkin et al, (1981) demonstrated that when the 
analysis accounted properly for the grouping of children into classes, the significant 
differences disappeared and the 'formally' taught children could not be shown to differ 
from the others. 

... In essence what was occurring here was that the children within any one 
classroom, because they were taught together, tended to be similar in their performance. 
As a result they provide rather less information than would have been the case if the 
same number of students had been taught separately by different teachers. In other 
words, the basic unit for purposes of comparison should have been the teacher not the 
student. The function of the students can be seen as providing, for each teacher, an 
estimate of that teacher's effectiveness. Increasing the number of students per teacher 
would increase the precision of those estimates but not change the number of teachers 
being compared. Beyond a certain point, simply increasing the numbers of students in 
this way hardly improves things at all. . . . 

Researchers have long recognised this issue. In education, for example, there has been 
much debate . . . about the so called 'unit of analysis' problem, which is the one just 
outlined. Before multilevel modelling became well developed as a research tool, the 
problems of ignoring hierarchical structures were reasonably well understood, but they 
were difficult to solve because powerful general purpose tools were unavailable. 



Models 1 and 3 (and the studies below by Cavalluzzo, by Goldhaber and Anthony, and by 
Vandevoort, et al.) use the traditional multiple regression approach, so that their conclusions 
are vulnerable to the same interpretation problem as outlined by Dr. Goldstein. Models 2 and 
4, by properly accounting for the nested structure of the data, produce more defensible results. 

The current study stands in contrast to other recent studies which reported that NBCTs 
are more likely to produce higher levels of student achievement than teachers who are not 
certified by the National Board. These studies include: 
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• Goldhaber and Anthony (March 2004). This study measured the effeetiveness of 
NBCTs by studying the annual test scores of North Carolina students in grades three, 
four and five from three aeademic years: 1996-97, 1997-98 and 1998-99. The study 
found that students of NBCTs experieneed year-end testing improvements that averaged 
7 percent to 15 pereent more than peers whose teachers were not NBCTs. 

• Cavalluzzo (2004) at the CNA Corporation found that students of NBCTs did a 
measurably better job than other ninth and tenth graders on year-end math tests, 
conditional on their previous year seore, in Miami-Dade County (Fla.) Public Schools. 

• Vandevoort, Amrein-Beardsley, and Berliner (2004), at Arizona State University, found 
that students of NBCTs outperformed students of non-NBCTs on the Stanford-9 
aehievement test, with learning gains equivalent on average to spending more than an 
extra month in sehool each year. 

Since none of these studies properly accounted for the nested structure of the data, their 
results are open to question. Caution is advised against too much reliance on the eonelusions 
which have been drawn from these reports until analyses are completed which use the 
appropriate sampling unit (i.e., teaehers within eertification status) as part of the error term to 
test the hypotheses of differences among NBPTS eertifioation status elasses. Additionally, it 
should be of interest to learn if the variability among individual teaehers within a eertification 
class is as large as indicated in the current study. 

It is worth noting that the eurrent study, and the other studies of the effeetiveness of 
NBPTS Certification cited above, use student achievement scores as the outcome measure. 
That is, these studies make the implieit assumption that growth in student achievement is an 
appropriate indieator of teaeher effeetiveness. Using this indicator, the eurrent study explieitly 
shows (and the small effeet sizes in earlier studies suggest that the same is true in those cases) 
that the eurrent NBPTS eertifieation process does a relatively poor job of distinguishing 
effeetive from ineffective teaehers. If growth in student aehievement is indeed an appropriate 
standard of teaeher effeetiveness, it follows that including student growth measures in the 
eertifieation process would vastly improve its ability to identify quality teaehers. 
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Table 1. Number of Teacher-Years and Student-Years used in the Analyses 





Certification Status 








Certified 






Future NBPTS 


Never in 








Teacher 


Failed NBPTS 


Candidate 


NBPTS 


All 




Teh 


Stu 


Teh 


Stu 


Teh 


Stu 


Teh 


Stu 


Teh 


Stu 




Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Yrs 


Subject 


Grade 






















Math 


4 


104 


2152 


46 


939 


92 


1882 


1616 


32241 


1858 


37214 




5 


122 


2592 


37 


775 


123 


2735 


1516 


30613 


1798 


36715 




6 


19 


1217 


14 


861 


19 


1337 


348 


20797 


400 


24212 




7 


13 


929 


11 


829 


17 


1462 


212 


13257 


253 


16477 




8 


23 


1834 


9 


736 


26 


2039 


242 


15510 


300 


20119 


Reading 


4 


104 


2152 


46 


939 


92 


1882 


1616 


32241 


1858 


37214 




5 


122 


2592 


37 


775 


123 


2735 


1516 


30613 


1798 


36715 




6 


32 


2210 


10 


685 


14 


967 


350 


19638 


406 


23500 




7 


25 


1925 


9 


773 


8 


705 


223 


13361 


265 


16764 




8 


23 


1737 


5 


405 


16 


1197 


253 


16500 


297 


19839 
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Table 2 Summary of Comparisons Between NBPTS Certified Teachers and Others 
Combined Data from Charlotte-Mecklenberg and Wake County 





Math 


Reading 


Model 


Model 


1 


2 


3 


4 


1 


2 


3 


4 


Grade 


Comparison 


+ 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


4 


Certified vs Faded NBPTS 


Certified vs Future Candidate 


++ 


ns 


+ 


ns 


ns 


ns 


ns 


ns 


Certtified vs Never in NBPTS 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


5 


Certified vs Failed NBPTS 


+ 


ns 


+ 


ns 


ns 


ns 


ns 


ns 


Certified vs Future Candidate 


++ 


+ 


++ 


+ 


ns 


ns 


ns 


ns 


Certtified vs Never in NBPTS 


++ 


ns 


++ 


ns 


++ 


+ 


ns 


ns 


6 


Certified vs Failed NBPTS 


++ 


ns 


+ 


ns 


+ 


ns 


ns 


ns 


Certified vs Future Candidate 


++ 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


Certtified vs Never in NBPTS 


++ 


ns 


ns 


ns 


++ 


++ 


ns 


ns 


7 


Certified vs Failed NBPTS 


ns 


ns 


ns 


ns 


- 


ns 


ns 


ns 


Certified vs Future Candidate 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


Certtified vs Never in NBPTS 


ns 


ns 


+ 


ns 


ns 


ns 


- 


- 


8 


Certified vs Failed NBPTS 


++ 


ns 


++ 


ns 


ns 


ns 


++ 


ns 


Certified vs Future Candidate 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


ns 


Certtified vs Never in NBPTS 


ns 


ns 


+ 


ns 


ns 


++ 


ns 


ns 



++ indicates that the NBPTS Certified Teacher estimate is significantly larger than the 
other, with p-value<.01. 

+ indicates that the NBPTS Certified Teacher estimate is significantly larger than the 
other, with .01<=p-value<.05. 

— indicates that the NBPTS Certified Teacher estimate is significantly smaller than the 
other, with p-value<.01. 

- indicates that the NBPTS Certified Teacher estimate is significantly smaller than the 
other, with .0K=p-value<.05. 

ns indicates that the NBPTS Certified Teacher estimate is not significantly different 
from the other. 



12 





Table 3A, Comparisons and Effects Sizes for Model 1 



Subject 


Grade 


Comparison 


Estimate 


Residual 

Variance 


Effect 

Size 


Math 


4 


Cert vs Failed 


0.34* 


19.04 


0.08 


Math 


4 


Cert vs Future 


0.46 ** 


19.04 


0.10 


Math 


4 


Cert vs Never 


0.19 


19.04 


0.04 


Math 


5 


Cert vs Failed 


0.44 * 


23.56 


0.09 


Math 


5 


Cert vs Future 


0.74 ** 


23.56 


0.15 


Math 


5 


Cert vs Never 


0.36 ** 


23.56 


0.07 


Math 


6 


Cert vs Failed 


0.99 ** 


20.26 


0.22 


Math 


6 


Cert vs Future 


0.54 ** 


20.26 


0.12 


Math 


6 


Cert vs Never 


0.56 ** 


20.26 


0.12 


Math 


7 


Cert vs Failed 


0.00 


24.85 


0.00 


Math 


7 


Cert vs Future 


0.09 


24.85 


0.02 


Math 


7 


Cert vs Never 


0.34 


24.85 


0.07 


Math 


8 


Cert vs Failed 


1.37 ** 


24.17 


0.28 


Math 


8 


Cert vs Future 


-0.07 


24.17 


-0.01 


Math 


8 


Cert vs Never 


0.24 


24.17 


0.05 


Reading 


4 


Cert vs Failed 


0.27 


26.26 


0.05 


Reading 


4 


Cert vs Future 


0.17 


26.26 


0.03 


Reading 


4 


Cert vs Never 


0.02 


26.26 


0.00 


Reading 


5 


Cert vs Failed 


0.26 


23.21 


0.05 


Reading 


5 


Cert vs Future 


0.06 


23.21 


0.01 


Reading 


5 


Cert vs Never 


0.28 ** 


23.21 


0.06 


Reading 


6 


Cert vs Failed 


0.50 * 


26.56 


0.10 


Reading 


6 


Cert vs Future 


0.35 


26.56 


0.07 


Reading 


6 


Cert vs Never 


0.60 ** 


26.56 


0.12 


Reading 


7 


Cert vs Failed 


-0.42 * 


23.22 


-0.09 


Reading 


7 


Cert vs Future 


0.21 


23.22 


0.04 


Reading 


7 


Cert vs Never 


-0.11 


23.22 


-0.02 


Reading 


8 


Cert vs Failed 


0.40 


23.29 


0.08 


Reading 


8 


Cert vs Future 


0.28 


23.29 


0.06 


Reading 


8 


Cert vs Never 


0.11 


23.29 


0.02 
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Table 3B, Comparisons and Effects Sizes for Model 2 



Subject 


Grade 


Comparison 


Estimate 


Residual 

Variance 


Effect 

Size 


Math 


4 


Cert vs Failed 


0.22 


17.03 


0.05 


Math 


4 


Cert vs Future 


0.22 


17.03 


0.05 


Math 


4 


Cert vs Never 


0.05 


17.03 


0.01 


Math 


5 


Cert vs Failed 


0.46 


20.83 


0.10 


Math 


5 


Cert vs Future 


0.64 * 


20.83 


0.14 


Math 


5 


Cert vs Never 


0.32 


20.83 


0.07 


Math 


6 


Cert vs Failed 


0.59 


17.73 


0.14 


Math 


6 


Cert vs Future 


0.07 


17.73 


0.02 


Math 


6 


Cert vs Never 


0.27 


17.73 


0.06 


Math 


7 


Cert vs Failed 


0.23 


22.34 


0.05 


Math 


7 


Cert vs Future 


0.44 


22.34 


0.09 


Math 


7 


Cert vs Never 


0.40 


22.34 


0.08 


Math 


8 


Cert vs Failed 


0.82 


22.01 


0.17 


Math 


8 


Cert vs Future 


-0.20 


22.01 


-0.04 


Math 


8 


Cert vs Never 


0.24 


22.01 


0.05 


Reading 


4 


Cert vs Failed 


0.17 


25.31 


0.03 


Reading 


4 


Cert vs Future 


0.06 


25.31 


0.01 


Reading 


4 


Cert vs Never 


-0.02 


25.31 


-0.00 


Reading 


5 


Cert vs Failed 


0.27 


22.41 


0.06 


Reading 


5 


Cert vs Future 


0.06 


22.41 


0.01 


Reading 


5 


Cert vs Never 


0.32 * 


22.41 


0.07 


Reading 


6 


Cert vs Failed 


0.46 


25.96 


0.09 


Reading 


6 


Cert vs Future 


0.12 


25.96 


0.02 


Reading 


6 


Cert vs Never 


0.58 ** 


25.96 


0.11 


Reading 


7 


Cert vs Failed 


-0.49 


22.51 


-0.10 


Reading 


7 


Cert vs Future 


0.15 


22.51 


0.03 


Reading 


7 


Cert vs Never 


0.05 


22.51 


0.01 


Reading 


8 


Cert vs Failed 


0.36 


22.66 


0.08 


Reading 


8 


Cert vs Future 


0.37 


22.66 


0.08 


Reading 


8 


Cert vs Never 


0.49 ** 


22.66 


0.10 



14 






Table 3C, Comparisons and Effects Sizes for Model 3 



Subject 


Grade 


Comparison 


Estimate 


Residual 

Variance 


Effect 

Size 


Math 


4 


Cert vs Failed 


0.26 


21.95 


0.05 


Math 


4 


Cert vs Future 


0.33 * 


21.95 


0.07 


Math 


4 


Cert vs Never 


0.11 


21.95 


0.02 


Math 


5 


Cert vs Failed 


0.46 * 


25.82 


0.09 


Math 


5 


Cert vs Future 


0.74 ** 


25.82 


0.15 


Math 


5 


Cert vs Never 


0.34 ** 


25.82 


0.07 


Math 


6 


Cert vs Failed 


0.49 * 


24.58 


0.10 


Math 


6 


Cert vs Future 


0.04 


24.58 


0.01 


Math 


6 


Cert vs Never 


-0.01 


24.58 


-0.00 


Math 


7 


Cert vs Failed 


-0.13 


26.28 


-0.03 


Math 


7 


Cert vs Future 


0.22 


26.28 


0.04 


Math 


7 


Cert vs Never 


0.46 * 


26.28 


0.09 


Math 


8 


Cert vs Failed 


1.09 ** 


29.60 


0.20 


Math 


8 


Cert vs Future 


-0.01 


29.60 


-0.00 


Math 


8 


Cert vs Never 


0.30 * 


29.60 


0.05 


Reading 


4 


Cert vs Failed 


0.06 


34.52 


0.01 


Reading 


4 


Cert vs Future 


-0.13 


34.52 


-0.02 


Reading 


4 


Cert vs Never 


-0.24 


34.52 


-0.04 


Reading 


5 


Cert vs Failed 


0.29 


31.36 


0.05 


Reading 


5 


Cert vs Future 


0.07 


31.36 


0.01 


Reading 


5 


Cert vs Never 


0.13 


31.36 


0.02 


Reading 


6 


Cert vs Failed 


0.51 


33.66 


0.09 


Reading 


6 


Cert vs Future 


-0.20 


33.66 


-0.03 


Reading 


6 


Cert vs Never 


0.22 


33.66 


0.04 


Reading 


7 


Cert vs Failed 


-0.28 


39.46 


-0.05 


Reading 


7 


Cert vs Future 


0.22 


39.46 


0.04 


Reading 


7 


Cert vs Never 


-0.46 ** 


39.46 


-0.07 


Reading 


8 


Cert vs Failed 


1.39 ** 


35.86 


0.23 


Reading 


8 


Cert vs Future 


0.13 


35.86 


0.02 


Reading 


8 


Cert vs Never 


-0.31 


35.86 


-0.05 
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Table 3D, Comparisons and Effects Sizes for Model 4 



Subject 


Grade 


Comparison 


Estimate 


Residual 

Variance 


Effect 

Size 


Math 


4 


Cert vs Failed 


0.16 


19.92 


0.04 


Math 


4 


Cert vs Future 


0.14 


19.92 


0.03 


Math 


4 


Cert vs Never 


-0.01 


19.92 


-0.00 


Math 


5 


Cert vs Failed 


0.53 


23.05 


0.11 


Math 


5 


Cert vs Future 


0.70 * 


23.05 


0.15 


Math 


5 


Cert vs Never 


0.29 


23.05 


0.06 


Math 


6 


Cert vs Failed 


0.28 


21.95 


0.06 


Math 


6 


Cert vs Future 


-0.30 


21.95 


-0.06 


Math 


6 


Cert vs Never 


-0.47 


21.95 


-0.10 


Math 


7 


Cert vs Failed 


0.13 


23.87 


0.03 


Math 


7 


Cert vs Future 


0.56 


23.87 


0.12 


Math 


7 


Cert vs Never 


0.56 


23.87 


0.12 


Math 


8 


Cert vs Failed 


0.54 


27.15 


0.10 


Math 


8 


Cert vs Future 


0.16 


27.15 


0.03 


Math 


8 


Cert vs Never 


0.05 


27.15 


0.01 


Reading 


4 


Cert vs Failed 


0.03 


33.65 


0.01 


Reading 


4 


Cert vs Future 


-0.15 


33.65 


-0.03 


Reading 


4 


Cert vs Never 


-0.22 


33.65 


-0.04 


Reading 


5 


Cert vs Failed 


0.31 


30.44 


0.06 


Reading 


5 


Cert vs Future 


0.03 


30.44 


0.01 


Reading 


5 


Cert vs Never 


0.09 


30.44 


0.02 


Reading 


6 


Cert vs Failed 


0.44 


32.64 


0.08 


Reading 


6 


Cert vs Future 


-0.44 


32.64 


-0.08 


Reading 


6 


Cert vs Never 


-0.04 


32.64 


-0.01 


Reading 


7 


Cert vs Failed 


-0.49 


37.86 


-0.08 


Reading 


7 


Cert vs Future 


0.10 


37.86 


0.02 


Reading 


7 


Cert vs Never 


-0.75 * 


37.86 


-0.12 


Reading 


8 


Cert vs Failed 


1.42 


35.03 


0.24 


Reading 


8 


Cert vs Future 


0.34 


35.03 


0.06 


Reading 


8 


Cert vs Never 


-0.24 


35.03 


-0.04 
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Teacher Effects from Model 2 without NBPTS Status 
Subject= Math Grade= 4 



Figure 1, Model 2 Teacher Effects for 4*'* Grade Math 




jaiioeei 
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Certification Status Certified Teacher Failed NBPTS ooo Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Math Grade= 5 



Figure 2, Model 2 Teacher Effects for S*** Grade Math 




ioejj.3 J8L|0^1 
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Certification Status * * Certified Teacher * * ^ Failed NBFTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Math Grade= 6 



Figure 3, Model 2 Teacher Effects for h*** Grade Math 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Math Grade= 7 



Figure 4, Model 2 Teacher Effects for T*** Grade Math 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Math Grade= S 



Figure 5, Model 2 Teacher Effects for S*** Grade Math 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Reading Grade=4 



Figure 6, Model 2 Teacher Effects for 4*** Grade Reading 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Reading Grade= 5 



Figure 7, Model 2 Teacher Effects for 5*** Grade Reading 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Reading Grade=6 



Figure 8, Model 2 Teacher Effects for 6*** Grade Reading 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Reading Grade=7 



Figure 9, Model 2 Teacher Effects for 7*** Grade Reading 
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Certification Status * * Certified Teacher * * ^ Failed NBPTS o o o Never in NBPTS 



Teacher Effects from Model 2 without NBPTS Status 
Subject= Reading Grade=8 



Figure 10, Model 2 Teacher Effects for S*** Grade Reading 
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Certification Status * * * Certified Teacher * * * Failed NBPTS o o o Never in NBPTS 



Table A.l Analysis Results for 4*'* Grade Mathematics 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


37184 


3.89 


** 


3 


1035 


0.34 


ns 


3 


37192 


1.95 


ns 


3 


1035 


0.26 


ns 


Gender of Student 


1 


37184 


43.70 


** 


1 


36149 


48.74 


** 


1 


37192 


1.03 


ns 


1 


36157 


1.07 


ns 


Prev Math Score(Year) 


4 


37184 


5018.34 


** 


4 


36149 


5304.73 


** 


















Prev Read Score(Year) 


4 


37184 


596.60 


** 


4 


36149 


628.31 


** 


















Race of Student 


5 


37184 


139.17 


** 


5 


36149 


147.76 


** 


5 


37192 


29.82 


** 


5 


36157 


33.43 


** 


Teacher Years Experience 


9 


37184 


13.59 


** 


9 


36149 


8.05 


** 


9 


37192 


8.29 


** 


9 


36157 


6.07 


** 


Year of Student Test 


3 


37184 


249.10 


** 


3 


36149 


251.72 


** 


3 


37192 


394.98 


** 


3 


36157 


309.15 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.34 


0.17 


2.00 


* 


0.22 


0.32 


0.68 


ns 


0.26 


0.18 


1.40 


ns 


0.16 


0.32 


0.50 


ns 


Certified vs Future NBPTS 
Candidate 


0.46 


0.14 


3.22 


** 


0.22 


0.30 


0.71 


ns 


0.33 


0.15 


2.15 


* 


0.14 


0.31 


0.46 


ns 


Certified vs Never in NBPTS 


0.19 


0.10 


1.89 


ns 


0.05 


0.21 


0.22 


ns 


0.11 


0.11 


1.00 


ns 


-0.01 


0.22 


-0.03 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




1.82 




1.84 


Failed NBPTS 




1.32 




1.23 


Future NBPTS Candidate 




2.70 




2.57 


Never in NBPTS 




2.19 




2.25 


Residual 


19.04 


17.03 


21.95 


19.92 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,2 Analysis Results for 5*** Grade Mathematics 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


36685 


9.84 


** 


3 


986 


1.48 


ns 


3 


36693 


9.10 


** 


3 


986 


1.68 


ns 


Gender of Student 


1 


36685 


11.04 


** 


1 


35699 


16.19 


** 


1 


36693 


4.62 


* 


1 


35707 


3.15 


ns 


Prev Math Score(Year) 


4 


36685 


6077.78 


** 


4 


35699 


6472.12 


** 


















Prev Read Score(Year) 


4 


36685 


504.72 


** 


4 


35699 


521.64 


** 


















Race of Student 


5 


36685 


117.76 


** 


5 


35699 


107.74 


** 


5 


36693 


65.41 


** 


5 


35707 


50.13 


** 


Teacher Years Experience 


9 


36685 


7.10 


** 


9 


35699 


1.94 


* 


9 


36693 


7.21 


** 


9 


35707 


1.75 


ns 


Year of Student Test 


3 


36685 


204.14 


** 


3 


35699 


199.91 


** 


3 


36693 


192.23 


** 


3 


35707 


180.46 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.44 


0.20 


2.19 


* 


0.46 


0.43 


1.07 


ns 


0.46 


0.21 


2.20 


* 


0.53 


0.46 


1.13 


ns 


Certified vs Future NBPTS 
Candidate 


0.74 


0.14 


5.40 


** 


0.64 


0.31 


2.06 


* 


0.74 


0.14 


5.15 


** 


0.70 


0.32 


2.17 


* 


Certified vs Never in NBPTS 


0.36 


0.10 


3.52 


** 


0.32 


0.22 


1.43 


ns 


0.34 


0.11 


3.16 


** 


0.29 


0.23 


1.24 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




2.44 




2.61 


Failed NBPTS 




3.00 




3.68 


Future NBPTS Candidate 




3.10 




3.46 


Never in NBPTS 




2.89 




2.89 


Residual 


23.56 


20.83 


25.82 


23.05 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,3 Analysis Results for 6**" Grade Mathematics 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


24188 


8.69 


** 


3 


291 


0.27 


ns 


3 


24192 


2.72 


* 


3 


291 


1.05 


ns 


Gender of Student 


1 


24188 


4.74 


* 


1 


23897 


4.91 


* 


1 


24192 


1.28 


ns 


1 


23901 


2.30 


ns 


Prev Math Score(Year) 


2 


24188 


10021.2 


** 


2 


23897 


10217.5 


** 


















Prev Read Score(Year) 


2 


24188 


511.12 


** 


2 


23897 


540.90 


** 


















Race of Student 


5 


24188 


82.96 


** 


5 


23897 


73.40 


** 


5 


24192 


26.00 


** 


5 


23901 


15.86 


** 


Teacher Years Experience 


9 


24188 


11.08 


** 


9 


23897 


3.27 


** 


9 


24192 


7.79 


** 


9 


23901 


3.87 


** 


Year of Student Test 


1 


24188 


174.51 


** 


1 


23897 


154.30 


** 


1 


24192 


0.25 


ns 


1 


23901 


0.06 


ns 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.99 


0.20 


4.93 


** 


0.59 


0.78 


0.76 


ns 


0.49 


0.22 


2.23 


* 


0.28 


0.77 


0.36 


ns 


Certified vs Future NBPTS 
Candidate 


0.54 


0.18 


2.98 


** 


0.07 


0.80 


0.08 


ns 


0.04 


0.20 


0.19 


ns 


-0.30 


0.80 


-0.38 


ns 


Certified vs Never in NBPTS 


0.56 


0.14 


4.07 


** 


0.27 


0.64 


0.43 


ns 


-0.01 


0.15 


-0.07 


ns 


-0.47 


0.66 


-0.72 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




5.10 




5.34 


Failed NBPTS 




2.15 




1.56 


Future NBPTS Candidate 




3.45 




2.95 


Never in NBPTS 




2.53 




2.87 


Residual 


20.26 


17.73 


24.58 


21.95 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,4 Analysis Results for 7*** Grade Mathematics 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


16453 


2.46 


ns 


3 


203 


0.22 


ns 


3 


16457 


4.86 


** 


3 


203 


0.58 


ns 


Gender of Student 


1 


16453 


2.20 


ns 


1 


16250 


2.35 


ns 


1 


16457 


32.67 


** 


1 


16254 


35.52 


** 


Prev Math Score(Year) 


2 


16453 


7639.82 


** 


2 


16250 


7437.94 


** 


















Prev Read Score(Year) 


2 


16453 


424.83 


** 


2 


16250 


442.72 


** 


















Race of Student 


5 


16453 


28.05 


** 


5 


16250 


25.63 


** 


5 


16457 


26.46 


** 


5 


16254 


14.40 


** 


Teacher Years Experience 


9 


16453 


11.60 


** 


9 


16250 


0.96 


ns 


9 


16457 


11.50 


** 


9 


16254 


0.89 


ns 


Year of Student Test 


1 


16453 


2.12 


ns 


1 


16250 


0.00 


ns 


1 


16457 


37.75 


** 


1 


16254 


60.25 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.00 


0.25 


0.02 


ns 


0.23 


0.65 


0.35 


ns 


- 0.13 


0.25 


- 0.53 


ns 


0.13 


0.68 


0.19 


ns 


Certified vs Future NBPTS 
Candidate 


0.09 


0.22 


0.44 


ns 


0.44 


0.61 


0.72 


ns 


0.22 


0.22 


0.99 


ns 


0.56 


0.63 


0.89 


ns 


Certified vs Never in NBPTS 


0.34 


0.18 


1.88 


ns 


0.40 


0.56 


0.71 


ns 


0.46 


0.19 


2.47 


* 


0.56 


0.59 


0.96 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




2.28 




2.54 


Failed NBPTS 




1.27 




1.36 


Future NBPTS Candidate 




0.83 




0.85 


Never in NBPTS 




3.01 




2.77 


Residual 


24.85 


22.34 


26.28 


23.87 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,5 Analysis Results for 8**" Grade Mathematics 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


20092 


16.35 


** 


3 


221 


0.66 


ns 


3 


20098 


8.52 


** 


3 


221 


0.20 


ns 


Gender of Student 


1 


20092 


29.03 


** 


1 


19871 


28.95 


** 


1 


20098 


0.94 


ns 


1 


19877 


0.17 


ns 


Prev Math Score(Year) 


3 


20092 


6688.87 


** 


3 


19871 


6466.84 


** 


















Prev Read Score(Year) 


3 


20092 


290.89 


** 


3 


19871 


288.69 


** 


















Race of Student 


5 


20092 


44.02 


** 


5 


19871 


33.71 


** 


5 


20098 


17.08 


** 


5 


19877 


12.81 


** 


Teacher Years Experience 


9 


20092 


24.28 


** 


9 


19871 


2.07 


* 


9 


20098 


15.85 


** 


9 


19877 


0.68 


ns 


Year of Student Test 


2 


20092 


35.28 


** 


2 


19871 


28.59 


** 


2 


20098 


14.96 


** 


2 


19877 


16.59 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


1.37 


0.22 


6.27 


** 


0.82 


0.72 


1.14 


ns 


1.09 


0.24 


4.50 


** 


0.54 


0.78 


0.70 


ns 


Certified vs Future NBPTS 
Candidate 


-0.07 


0.17 


-0.39 


ns 


-0.20 


0.69 


-0.30 


ns 


-0.01 


0.19 


-0.04 


ns 


0.16 


0.71 


0.23 


ns 


Certified vs Never in NBPTS 


0.24 


0.13 


1.89 


ns 


0.24 


0.46 


0.53 


ns 


0.30 


0.14 


2.11 


* 


0.05 


0.47 


0.11 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




2.82 




2.96 


Failed NBPTS 




2.41 




2.87 


Future NBPTS Candidate 




4.68 




4.94 


Never in NBPTS 




2.81 




2.86 


Residual 


24.17 


22.01 


29.60 


27.15 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,6 Analysis Results for 4**" Grade Reading 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


37184 


1.22 


ns 


3 


1035 


0.26 


ns 


3 


37192 


1.83 


ns 


3 


1035 


0.75 


ns 


Gender of Student 


1 


37184 


113.32 


** 


1 


36149 


116.66 


** 


1 


37192 


16.09 


** 


1 


36157 


16.59 


** 


Prev Math Score(Year) 


4 


37184 


876.72 


** 


4 


36149 


864.47 


** 


















Prev Read Score(Year) 


4 


37184 


3563.78 


** 


4 


36149 


3545.80 


** 


















Race of Student 


5 


37184 


108.06 


** 


5 


36149 


92.75 


** 


5 


37192 


11.58 


** 


5 


36157 


11.31 


** 


Teacher Years Experience 


9 


37184 


14.94 


** 


9 


36149 


8.11 


** 


9 


37192 


8.21 


** 


9 


36157 


4.58 


** 


Year of Student Test 


3 


37184 


121.24 


** 


3 


36149 


113.89 


** 


3 


37192 


23.28 


** 


3 


36157 


18.83 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.27 


0.20 


1.33 


ns 


0.17 


0.29 


0.61 


ns 


0.06 


0.23 


0.27 


ns 


0.03 


0.30 


0.11 


ns 


Certified vs Future NBPTS 
Candidate 


0.17 


0.17 


1.04 


ns 


0.06 


0.24 


0.23 


ns 


- 0.13 


0.19 


- 0.67 


ns 


- 0.15 


0.26 


- 0.56 


ns 


Certified vs Never in NBPTS 


0.02 


0.12 


0.14 


ns 


- 0.02 


0.18 


- 0.08 


ns 


- 0.24 


0.13 


- 1.77 


ns 


- 0.22 


0.19 


- 1.16 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




0.99 




0.85 


Failed NBPTS 




0.73 




0.65 


Future NBPTS Candidate 




0.77 




1.05 


Never in NBPTS 




1.04 




0.91 


Residual 


26.26 


25.31 


34.52 


33.65 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A.7 Analysis Results for 5**" Grade Reading 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


36685 


3.86 


** 


3 


986 


2.12 


ns 


3 


36693 


0.73 


ns 


3 


986 


0.42 


ns 


Gender of Student 


1 


36685 


120.08 


** 


1 


35699 


123.67 


** 


1 


36693 


2.72 


ns 


1 


35707 


2.59 


ns 


Prev Math Score(Year) 


4 


36685 


615.95 


** 


4 


35699 


618.80 


** 


















Prev Read Score(Year) 


4 


36685 


4001.30 


** 


4 


35699 


3947.45 


** 


















Race of Student 


5 


36685 


92.32 


** 


5 


35699 


77.36 


** 


5 


36693 


54.59 


** 


5 


35707 


38.79 


** 


Teacher Years Experience 


9 


36685 


9.17 


** 


9 


35699 


4.85 


** 


9 


36693 


2.91 


** 


9 


35707 


2.30 


* 


Year of Student Test 


3 


36685 


229.57 


** 


3 


35699 


225.70 


** 


3 


36693 


15.42 


** 


3 


35707 


14.19 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.26 


0.20 


1.33 


ns 


0.27 


0.27 


1.01 


ns 


0.29 


0.23 


1.26 


ns 


0.31 


0.29 


1.06 


ns 


Certified vs Future NBPTS 
Candidate 


0.06 


0.14 


0.44 


ns 


0.06 


0.20 


0.30 


ns 


0.07 


0.16 


0.42 


ns 


0.03 


0.23 


0.14 


ns 


Certified vs Never in NBPTS 


0.28 


0.10 


2.76 


** 


0.32 


0.16 


2.04 


* 


0.13 


0.12 


1.10 


ns 


0.09 


0.17 


0.51 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




0.81 




0.96 


Failed NBPTS 




0.56 




0.43 


Future NBPTS Candidate 




0.74 




0.74 


Never in NBPTS 




0.87 




1.02 


Residual 


23.21 


22.41 


31.36 


30.44 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A,8 Analysis Results for 6**" Grade Reading 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


23476 


8.40 


** 


3 


298 


2.90 


* 


3 


23480 


2.97 


* 


3 


298 


1.01 


ns 


Gender of Student 


1 


23476 


197.92 


** 


1 


23178 


196.33 


** 


1 


23480 


38.16 


** 


1 


23182 


42.04 


** 


Prev Math Score(Year) 


2 


23476 


1086.92 


** 


2 


23178 


1032.85 


** 


















Prev Read Score(Year) 


2 


23476 


4822.65 


** 


2 


23178 


4608.12 


** 


















Race of Student 


5 


23476 


68.95 


** 


5 


23178 


52.94 


** 


5 


23480 


4.89 


** 


5 


23182 


3.41 


** 


Teacher Years Experience 


9 


23476 


4.82 


** 


9 


23178 


1.68 


ns 


9 


23480 


5.09 


** 


9 


23182 


3.49 


** 


Year of Student Test 


1 


23476 


1628.89 


** 


1 


23178 


1425.54 


** 


1 


23480 


624.11 


** 


1 


23182 


443.86 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.50 


0.23 


2.14 


* 


0.46 


0.34 


1.36 


ns 


0.51 


0.26 


1.94 


ns 


0.44 


0.44 


1.00 


ns 


Certified vs Future NBPTS 
Candidate 


0.35 


0.21 


1.69 


ns 


0.12 


0.37 


0.33 


ns 


-0.20 


0.23 


-0.85 


ns 


-0.44 


0.41 


-1.07 


ns 


Certified vs Never in NBPTS 


0.60 


0.12 


4.91 


** 


0.58 


0.21 


2.73 


** 


0.22 


0.14 


1.63 


ns 


-0.04 


0.25 


-0.16 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




0.47 




0.66 


Failed NBPTS 




0.25 




0.67 


Future NBPTS Candidate 




0.85 




0.99 


Never in NBPTS 




0.79 




1.48 


Residual 


26.56 


25.96 


33.66 


32.64 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A.9 Analysis Results for 7*** Grade Reading 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


16740 


2.26 


ns 


3 


213 


2.25 


ns 


3 


16744 


4.45 


** 


3 


213 


2.62 


ns 


Gender of Student 


1 


16740 


96.23 


** 


1 


16527 


90.77 


** 


1 


16744 


0.15 


ns 


1 


16531 


0.03 


ns 


Prev Math Score(Year) 


2 


16740 


626.36 


** 


2 


16527 


605.73 


** 


















Prev Read Score(Year) 


2 


16740 


3553.00 


** 


2 


16527 


3397.04 


** 


















Race of Student 


5 


16740 


18.77 


** 


5 


16527 


16.05 


** 


5 


16744 


138.81 


** 


5 


16531 


77.76 


** 


Teacher Years Experience 


9 


16740 


12.05 


** 


9 


16527 


3.79 


** 


9 


16744 


1.82 


ns 


9 


16531 


0.47 


ns 


Year of Student Test 


1 


16740 


180.94 


** 


1 


16527 


174.37 


** 


1 


16744 


8.47 


** 


1 


16531 


6.23 


* 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


- 0.42 


0.21 


- 1.99 


* 


- 0.49 


0.31 


- 1.59 


ns 


- 0.28 


0.27 


- 1.04 


ns 


- 0.49 


0.61 


- 0.82 


ns 


Certified vs Future NBPTS 
Candidate 


0.21 


0.22 


0.96 


ns 


0.15 


0.46 


0.32 


ns 


0.22 


0.28 


0.78 


ns 


0.10 


0.55 


0.17 


ns 


Certified vs Never in NBPTS 


- 0.11 


0.13 


- 0.89 


ns 


0.05 


0.28 


0.19 


ns 


- 0.46 


0.17 


- 2.80 


** 


- 0.75 


0.29 


- 2.58 


* 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




0.85 




0.47 


Failed NBPTS 




0.00 




2.05 


Future NBPTS Candidate 




0.76 




1.31 


Never in NBPTS 




1.03 




2.40 


Residual 


23.22 


22.51 


39.46 


37.86 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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Table A. 10 Analysis Results for 8*** Grade Reading 





Response Variable=Scale Score 


Response Variable=Simple Seale Score Gain 


Model 1 


Model 2 


Model 3 


Model 4 


Eixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Fixed Effects Only 


Fixed Effects and Random 
Teacher(Certification Status) 


Effect 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Num 

DF 


Den 

DF 


F- 

Value 


p- 

Value 


Certification Status 


3 


19812 


1.24 


ns 


3 


218 


2.42 


ns 


3 


19818 


12.03 


** 


3 


218 


2.71 


* 


Gender of Student 


1 


19812 


105.34 


** 


1 


19594 


100.58 


** 


1 


19818 


0.24 


ns 


1 


19600 


0.47 


ns 


Prev Math Score(Year) 


3 


19812 


696.71 


** 


3 


19594 


666.41 


** 


















Prev Read Score(Year) 


3 


19812 


2626.00 


** 


3 


19594 


2471.82 


** 


















Race of Student 


5 


19812 


48.19 


** 


5 


19594 


39.95 


** 


5 


19818 


37.76 


** 


5 


19600 


20.90 


** 


Teacher Years Experience 


9 


19812 


6.13 


** 


9 


19594 


3.14 


** 


9 


19818 


3.69 


** 


9 


19600 


2.56 


** 


Year of Student Test 


2 


19812 


113.84 


** 


2 


19594 


97.42 


** 


2 


19818 


28.20 


** 


2 


19600 


24.73 


** 




Comparisons 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Est. 


SE 


t- 

Value 


P- 

value 


Certified vs Failed NBPTS 


0.40 


0.27 


1.50 


ns 


0.36 


0.28 


1.28 


ns 


1.39 


0.33 


4.20 


** 


1.42 


0.86 


1.64 


ns 


Certified vs Future NBPTS 
Candidate 


0.28 


0.19 


1.47 


ns 


0.37 


0.30 


1.25 


ns 


0.13 


0.24 


0.57 


ns 


0.34 


0.42 


0.81 


ns 


Certified vs Never in NBPTS 


0.11 


0.13 


0.80 


ns 


0.49 


0.18 


2.68 


** 


-0.31 


0.16 


-1.91 


ns 


-0.24 


0.36 


-0.66 


ns 




Variance Component 
Estimates 


Estimate 


Estimate 


Estimate 


Estimate 


Certified Teacher 




0.08 




1.31 


Failed NBPTS 




0.00 




2.11 


Future NBPTS Candidate 




0.49 




0.32 


Never in NBPTS 




1.00 




1.12 


Residual 


23.29 


22.66 


35.86 


35.03 



* indicates statistical significance at the 0.05 level; ** indicates statistical significance at the 0.01 level; ns means not significant. 
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